Real-Time Speech-Driven Face Animation
نویسندگان
چکیده
This chapter presents our research on real-time speech-driven face animation. First, a visual representation, called Motion Unit (MU), for facial deformation is learned from a set of labeled face deformation data. A facial deformation can be approximated by a linear combination of MUs weighted by the corresponding MU parameters (MUPs), which are used as the visual features of facial deformations. MUs explore the correlation among those facial feature points used by the MPEG-4 face animation (FA) to describe facial deformations. MU-based FA is compatible with MPEG-4 FA. We then collect a set of audio-visual (AV) training database and use the training database to train a real-time audio-to-visual mapping (AVM).
منابع مشابه
Real-time speech-driven face animation with expressions using neural networks
A real-time speech-driven synthetic talking face provides an effective multimodal communication interface in distributed collaboration environments. Nonverbal gestures such as facial expressions are important to human communication and should be considered by speech-driven face animation systems. In this paper, we present a framework that systematically addresses facial deformation modeling, au...
متن کاملReal-time Speech Driven Avatar with Constant Short Time Delay
It is shown that the perception of speech is inherently multimodal [16][22]. Auditory-visual speech recognition is more accurate than auditory only or visual only speech recognition [1][10]. Research shows that a synthetic talking face can help people understand the associated speech in noisy environments [16]. It also helps people react more positively in interactive services [20]. In some sit...
متن کاملReal-time Face Animation Driven by Human Voice
-This paper presents a review of the recent research that examines the problem of generating full facial animation from auditory input speech signal. The major applications of this technique include face animation, human-computer interfaces, computer-aided instruction, video games and multimedia telephony for the hearing impaired. Using human voice for the real-time animation of synthetic faces...
متن کاملAn Integrated Framework for Face Modeling , Facial Motion
This dissertation presents an integrated framework for face modeling, facial motion analysis, and facial motion synthesis. This framework systematically addresses three closely related research issues: (1) selecting a quantitative representation of facial deformation for face modeling and animation; (2) automatic facial motion analysis based on the same visual representation; and (3) speech-to-...
متن کاملDynamic Mapping Method Based Speech Driven Face Animation System
In the paper, we design and develop a speech driven face animation system based on the dynamic mapping method. The face animation is synthesized by the unit concatenating, and synchronous with the real speech. The units are selected according to the cost functions which correspond to voice spectrum distance between training and target units. Visual distance between two adjacent training units i...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2001